Papers by Zoha Hayat Bhatti
Plasticity vs. Rigidity: The Impact of Low-Rank Adapters on Reasoning on a Micro-Budget (2026.eacl-srw)
Copied to clipboard
| Challenge: | Recent advances in mathematical reasoning typically rely on massive scale . yet, can strong reasoning capabilities be induced in small language models under extreme constraints? |
| Approach: | They train small language models with a single GPU for under 24 hours . they find that adapters unlock significant plasticity in standard instruction-tuned models . |
| Outcome: | The proposed model training on a single GPU (48GB) achieves 40% Pass@1 on AIME 24 (an 11.1% improvement over baseline) the model training results show that the adapter capacity and initialization are critical factors. |